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Technology  Readiness  Level  only  tells  part  of  the  story  of 
system  maturation.  As  component  technologies  are  devel¬ 
oped  to  become  part  of  systems,  there  are  also  integration 
and  manufacturing  issues  to  consider.  This  article  improves 
upon  the  System  and  Integration  Readiness  Level  concepts 
previously  developed  by  B.  J.  Sauser  et  al.,  combines  the 
concepts  of  Technology,  Integration,  and  Manufac 
turing  Readiness  Levels,  adapted  for  use  in  defense 
acquisition,  into  a  single  metric— System  Readiness 
Level.  This  metric  can  then  be  used  as  an  indicator  ' 
to  identify  areas  for  resource  allocation  to  enable 
the  most  efficient  path  to  technology  transition 
and  to  prevent  premature  system  advancement. 
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In  an  ideal  world,  a  component  technology  would  develop  concurrently 
with  its  interfaces  and  its  ability  to  be  manufactured.  In  the  real  world, 
technologies  lead  both  their  interfaces  and  manufacturing  infrastructure. 
For  example,  motorcycles  were  first  made  with  fixed  foot-pegs  until  some 
rather  spectacular,  spin-out  wrecks  occurred,  prompting  folding  foot-pegs. 
The  human-motorcycle  interface  maturity  followed  the  technical  maturity 
at  the  expense  of  safety.  Early  airplanes  were  made,  one-at-a-time,  with 
bicycle  manufacturing  equipment.  The  manufacturing  maturity  lagged  the 
technology.  The  competing  pitfall  in  system  development  is  the  premature 
advancement  of  a  technology  to  the  next  level  of  development  in  advance  of 
its  interfaces,  such  as  the  current  state  of  the  F-35  program.  Although  the 
program  is  in  late  stage  development,  interface  and  component  technology 
issues  are  still  emerging  that  are  preventing  full  operational  capability 
(Bender,  2015).  We  can  do  a  better  job  by  minimizing  the  gap  between 
interface,  manufacturing,  and  technology  maturity.  Integration  and  sys¬ 
tem  readiness  are  not  yet  implemented  in  any  formal  way  Department  of 
Defense  (DoD)-wide. 

This  article  explains  a  method  to  combine  Technology  Readiness  Level 
(TRL)  (See  Appendix,  Table  A-l),  Integration  Readiness  Level  (IRL),  and 
Manufacturing  Readiness  Level  (MRL)  (See  Appendix,  Table  A- 2)  into  a 
single  metric— System  Readiness  Level  (SRL)— that  can  provide  guidance  to 
decision  makers  during  the  technology  maturation  process.  Such  guidance 
can  minimize  the  delays  and  mishaps  likely  to  occur  when  interfaces  and 
manufacturing  significantly  lag  their  component  technologies. 


Background 

The  DoD  Research,  Development,  Test  and  Evaluation  budget  is  sub¬ 
divided  into  seven  separate  activities:  basic  research;  applied  research; 
advanced  technology  development;  advanced  component  development  and 
prototypes;  system  development  and  demonstration;  research,  develop¬ 
ment,  test  and  evaluation  (RDT&E)  management  support;  and  operational 
systems  development,  i.e.,  the  DoD  categories  of  funding  and  technology 
development  (Appendix,  Table  A- 3).  These  seven  activities  are  designated 
as  DoD  6.1  through  6.7.  This  article  incorporates  the  6.1  through  6.7  levels 
of  funding  and  appropriate  levels  of  maturity  so  that  the  same  metric  can  be 
used  throughout  the  acquisition  life  cycle.  Verbal  definitions  of  TRL,  MRL, 
IRL,  and  SRL  are  included  at  the  end  of  the  article. 
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Sauser,  Ramirez-Marquez,  and  Devanandham,  and  Dimarzio  (2007),  and 
Sauser,  Ramirez-Marquez,  Magnaye,  and  Tan  (2008a)  furthered  the  con¬ 
cepts  of  TRL  to  include  IRL  and  SRL  (Sauser,  Forbes,  Long,  &  McGrory, 
2009;  Sauser,  Gove,  Forbes,  &  Ramirez-Marquez,  2010).  These  approaches 
emphasize  that  the  interfaces  between  subsystems  are  every  bit  as  import¬ 
ant  as  the  subsystems  themselves,  and  that  no  system  can  be  deemed  ready 
for  deployment  based  on  the  component  technologies  alone. 


Method 

Sauser's  basic  approach  is  to  imagine  a  system  composed  of  component 
technologies  from  1  to  n,  each  with  a  TRL  as  shown  in  equation  (1)  and 
Figure  1. 


TRL  =  trl  =  {trl  trL  ■■■  trl  } 

x  ^  1  2  nJ 


(l) 


Mathematical  Note.  A  list  of  symbols  or  numbers  in  braces 
represents  a  vector.  A  subscripted  symbol  indicates  one 
element  out  of  a  vector.  A  number  without  subscripts  indi¬ 
cates  the  whole  vector  quantity.  Lower  case  is  used  for 
normalized  quantities. 
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For  example,  a  motorcycle  can  be  viewed  as  an  engine,  power  train,  exhaust, 
electrical  system,  cooling  system,  saddle,  suspension,  wheels,  gauges,  steer¬ 
ing,  headlamp,  etc. 

Each  component  technology  has  a  potential  interface  with  each  other  com¬ 
ponent  and  with  the  external  environment,  including  the  possibility  of  an 
interface  going  both  ways,  as  shown  in  equation  (2).  For  simplicity,  Figure  2 
shows  the  interfaces  with  double  arrows,  as  if  irl12  =  irl21,  which  need  not  be 
the  case.  IRL  must  be  expressed  as  a  two-dimensional  matrix  rather  than 
a  one-dimensional  vector.  The  vector  is  generally  square— with  the  same 
number  of  rows  and  columns.  The  diagonal  of  the  matrix  is  not  used  since 
a  technology  always  works  with  itself. 
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In  the  Sauser  approach,  The  IRL  matrix  and  the  TRL  vector  are  multiplied 
together  as  a  vector  product  (U.S.  Navy,  2009,  p.  35)  to  form  an  SRL  vector 
that  can  be  averaged  for  an  overall  SRL  (Sauser,  Verma,  Ramirez-Marequez, 
Gove,  2006,  p.  A-12;  Sauser  et  al.,  2007,  p.  681;  U.S.  Navy,  2009,  p.  33).  Note 
that  this  paper  shows  matrix  notation  in  both  reduced  tensor  notation  and 
matrix  notation  as  a  convenience  for  a  multidisciplinary  audience.  SRL., 
[SRL]  and  SRL  all  refer  to  the  same  vector  entity  and  all  versions  of  equation 
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(3)  show  the  same  tensor/matrix  operation  in  different  notation.  Equation 

(4)  shows  the  Sauser  formula  for  SRL.  Computational  and  practical  exam¬ 
ples  of  all  formulas  will  be  shown  in  the  examples  section. 


SRL.  =  IRL..  TRV  =  IRL1.  TRL 1  +  IRL0.  TRL 2  +  +  =  IRL  .  TRL 

J  ij  J  2 j  nj 


(3a) 
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SRL=  *\ 
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/ 

k  trl  j 

(3c) 


SRL  =  4  Z  srl 


Nj-A 


j  =  1  to  N 


(4) 


As  shown  in  equations  (3a)  and  (3b),  the  Sauser  mathematics  views  a  com¬ 
ponent  of  SRL  (SRL. )  as  being  based  upon  a  single  interface  type  and  its 
associated  technologies;  the  SRL±  component  includes  TRL±,  TRL2,  etc., 
and  all  of  the  IRLln  rather  than  a  technology-centric  approach 
that  included  TRL±  with  all  its  interfaces.  The  inter¬ 
face-centric  approach  is  graphically  shown  in  Figure  3 
and  contrasted  with  a  technology-centric  approach  in 
Figure  4  using  a  motorcycle.  The  mechanical  compo¬ 
nent  of  SRL  (SRL  ,  .  n)  in  the  Sauser 
approach  for  a  motorcycle  would  be 
based  upon  the  mechanical-en¬ 
gine,  mechanical-headlamps, 
mechanical-saddle,  mechan¬ 
ical-tires,  etc.,  interfaces.  The 
interface-centric  approach  has 
some  serious  limitations  as  will 
be  covered  in  the  next  sections. 
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Note.  (Sauser  et  al.,  2007) 


FIGURE  4.  LEFT:  EXAMPLE  FROM  A  MOTORCYCLE:  INTERFACE¬ 
CENTRIC  APPROACH;  RIGHT:  TECHNOLOGY-CENTRIC  APPROACH 
PRESENTED  IN  THIS  ARTICLE 


Note.  Left:  (Sauser,  2008).  Right:  Ross,  S.  (2016).  Application  of  System  and  Integration 
Readiness  Levels  to  Department  of  Defense  Research  and  Development.  Defense 
Acquisition  Research  Journal,  23(7)),  In  Print. 


The  average  of  the  SRL  vector,  equation  (4),  describes  how  mature  the  sys¬ 
tem  is.  The  Sauser  approach  may  make  sense  for  a  single  mission  or  project, 
such  as  the  deployment  of  a  new  software  system.  However,  it  has  some 
serious  drawbacks  for  use  in  research  and  development  where  planners 
need  to  decide  what  technologies  to  develop  for  the  eventual  deployment  of 
a  new  platform,  weapon,  or  system.  First,  SRL,  as  defined  in  the  U.S.  Navy's 
Littoral  Combat  Ship  Mission  Module  Program  System  Maturity  Assessment 
Guide  (2009),  is  interface- centric  as  opposed  to  component- centric.  The 
Sauser  definition  shows  each  interface  with  its  associated  technologies 
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rather  than  each  technology  with  its  associated  interfaces.  Second,  SRL  as 
defined  by  Sauser,  has  no  clear  meaning  assigned  to  a  given  numerical  value. 
In  one  presentation  (Sauser,  Ramirez-Marquez,  Magnaye,  &  Tan,  2008b), 
SRL  is  defined  along  a  value  from  0  to  1  with  five  unequal  intermediate  levels 
and  no  verbal  definitions  akin  to  those  for  TRL,  IRL,  and  MRL.  This  gives 
SRL  a  different  kind  of  scale  than  IRL  and  TRL,  which  are  clearly  defined 
such  that  1  is  a  concept  and  9  is  full  deployment.  Third,  the  Sauser-defined 
SRL  only  has  meaning  at  the  full  system  level.  The  interface-centric  compo¬ 
nents  of  the  SRL  vector  give  no  guidance  to  component  developers.  Finally, 
the  definitions  of  IRL  tend  to  be  information  technology  (IT)-centric, 
emphasizing  control  and  information.  IRL  needs  to  be  applicable  to  a  wide 
variety  of  interfaces,  including  mechanical,  thermal,  electrical,  structural, 
and  control  interfaces  as  well  as  logistics,  policy,  and  other  ‘-ility*  and  mis¬ 
sion  interfaces. 

Characteristics  of  a  Useful  System  Readiness  Level  Metric 

A  useful  metric  will  be  defined  so  as  to  give  a  clear  indication  for  plan¬ 
ning  resource  allocation.  SRL  and  IRL,  as  metrics,  can  be  useful  if  they  are 
defined  correctly.  The  author  proposes  the  following  criteria  for  a  useful 
SRL  and  IRL  metric. 

1.  IRL  definitions  should  be  applicable  to  a  wide  variety  of 
technologies. 

2.  SRL  should  be  defined  such  that  SRL=1  is  a  concept  and  SRL=9 
is  a  mature,  deployed  system  on  the  same  basic  scale  as  TRL, 
MRL,  and  IRL. 

3.  SRL  should  equal  TRL  when  the  interfaces  are  developed  con¬ 
currently  with  the  components,  and  should  be  less  than  TRL 
when  interfaces  are  less  mature  than  the  components.  This 
will  give  planners  a  clear  metric  that  lets  them  know  that  it  is 
time  to  transition  funding  into  more  interface-centric  devel¬ 
opment  or  to  proceed  with  component  technology  maturation. 

4.  SRL  should  be  technology-  or  component- centric,  not 
interface- centric.  This  makes  it  clear  when  a  particular 
subcomponent  is  not  able  to  progress  further  toward  imple¬ 
mentation  due  to  an  interface  or  manufacturing  issue. 

5.  SRL  should  include  MRL,  TRL,  and  IRL. 


Defense ARJ,  July  2016,  Vol.  23 No.  3: 248-273 

Approved  for  public  release,  distribution  unlimited. 


255 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


6.  SRL  should  be  applicable  to  a  wide  variety  of  technical  matur¬ 
ities  (see  Appendix,  Table  A- 3),  including  basic  research  (6.1 
funding);  applied  research  (6.2  funding);  advanced  technology 
development  (6.3  funding);  advanced  component  develop¬ 
ment  and  prototypes  (6.4  funding);  system  development  and 
demonstration  (6.5  funding);  and  operational  systems  devel¬ 
opment  (6.7  funding),  i.e.,  the  DoD  categories  of  funding  and 
technology  development.  Note  that  6.6  funding  is  not  included 
because  it  is  for  management  activities  and  not  tied  to  a  level 
of  technical  maturity. 

7.  SRL  must  be  defined  in  such  a  way  as  to  avoid  maturity  in  one 
component  overshadowing  immaturity  in  another  (Kujawski, 
2010)  and  giving  the  illusion  that  the  system  is  ready  to  prog¬ 
ress.  This  implies  that  SRL  should  never  be  able  to  be  greater 
than  TRL  at  either  the  system  or  component  level. 


Proposed  System  Readiness  Level  Metric 

The  author  proposes  that  a  more  useful  way  to  arrange  MRL,  TRL,  and 
IRL  is  as  a  series  of  normalized  dot  products,  rather  than  vector  products 
(Sauser  et  al.,  2008a,  p.  47).  This  changes  the  view  of  the  components  of 
SRL  from  being  interface-centric  to  being  technology-centric,  as  shown 
in  the  contrast  between  Figure  3  and  Figure  5,  and  between  the  right  and 
left  sides  of  Figure  4.  The  SRL  components  are  equal  to  the  product  of  the 
normalized  MRL,  the  TRL,  and  the  mean  of  the  normalized  IRL, 
as  shown  in  Table  1.  In  the  notation  that  follows,  upper  case  is 
reserved  for  standard  (i.e.,  verbal)  definitions  and  lower  case 
is  for  normalized  quantities.  Note  that  the  word  'system' 
in  this  article  refers  to  a  generic  system— anything  that 
can  be  usefully  viewed  as  being  composed  of  parts,  rather 
than  specifically  as  a  deployed  military  asset.  Likewise, 
the  term  'component'  refers  to  the  parts  that  make  up 
a  larger  grouping  rather  than  exclusively  as  a  line -re¬ 
placeable  item  with  a  specific  part  number.  The  term 
interface  should  be  viewed  in  the  broad  sense  of  the 
word  to  also  include  the  external  environment— the 
'ilities'  (availability,  maintainability,  vulnerability, 
reliability,  supportability,  etc.)  and  the  DOTmLPF-P 
(Doctrine,  Organization,  Training,  materiel,  Leadership 
and  Education,  Personnel,  Facilities-Policy). 
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FIGURE  5.  A  COMPONENT  OF  SRL  BASED  UPON  A  SINGLE 
TECHNOLOGY  AND  ITS  INTERFACES 


Note.  Ross,  S.,  (2016).  Application  of  System  and  Integration  Readiness  Levels  to 
Department  of  Defense  Research  and  Development.  Defense  Acquisition  Research 

Journal,  In  Print. 

TABLE  1.  NORMALIZED  INTEGRATION  READINESS 

LEVEL  DEFINITIONS 

IRL  = 

Integration  readiness  level  scalar 

IRL  for  the  interface  between  technology  j  and  technology  k 

iHik  = 

normalized  IRL  for  interface  between  technology  j  and 
technology  k 

irl  = 

_ u _ 

1,  the  interface  always  works  with  itself 

irlkj,  the  interface  works  both  ways.  It  may  be  useful  for  some 
.  I  _  systems  to  break  the  IRL  apart  into  two  components.  For 

jk  purposes  of  this  article,  the  author  assumes  that  if  the  motor- 

fuel  interface  works,  so  does  the  fuel-motor  interface. 


irl  = 

IRL/i* 

Research 

level 

6.1 

6.2 

6.3 

6.4  6.5 

6.6 

6.7 

i* 

1 

2 

3 

5  6 

7 

9 

n 

iri  =  meanlirl^  f  ^  irlii 

y  =  i 

To  have  SRL  equal  to  TRL  when  IRL  and  MRL  are  at  commensurate  levels 
of  development  requires  normalized  versions  of  IRL  and  MRL  scaled  to 
the  level  of  research.  Basic  research  (6.1  funding)  should  have  a  goal  of  an 
IRL  of  1  (Interface  identification)  and  MRL  of  2  (Manufacturing  concepts 
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identified),  so  that  the  normalized  mrl  and  irl  equal  1  when  the  appropriate 
levels  of  IRL  and  MRL  are  reached.  Likewise,  system  development  and 
demonstration  (6.5  funding)  should  have  as  its  goal  an  IRL  of  6  (interface 
control)  and  an  MRL  of  6  (prototype  in  a  production-relevant  environ¬ 
ment)  so  the  normalized  mrl  and  irl  equal  1  when  the  appropriate  levels 
are  reached.  The  signal  to  proceed  to  the  next  step  in  system  development 
occurs  when  SRL  equals  TRL,  indicating  that  the  interfaces  and  manufac¬ 
turing  base  are  at  a  commensurate  level  of  development  with  the  component 
technologies.  The  nomenclature  and  definitions  for  normalized  IRL  are 
shown  in  Table  1.  The  normalization  factors  are  chosen  to  be  consistent  with 
the  funding  categories  listed  in  the  DoD  Research,  Development,  Test  and 
Evaluation  (RDT&E)  budget  (Appendix,  Table  A- 3).  Different  communities 
may  have  differing  levels  of  MRL,  TRL,  and  IRL  goals  vs.  acquisition  stage 
so  that  the  normalization  factors  are  intended  as  starting  suggestions.  It 
would  also  be  viable  to  have  normalization  factors  based  on  the  DoD  5000.02 
Model  1  (DoD,  2015). 

The  normalized  IRLs  associated  with  a  particular  technology  need  to  be 
averaged  to  come  up  with  a  representative  number  indicating  how  well 
that  particular  technology  relates  to  the  other  subsystems  or  technologies 
in  the  system.  The  irl.  accomplish  this.  Note  that  the  normalization  factors 
'reset'  the  metric  at  each  level  of  maturity,  which  reduces  the  possibility  of 
one  very  mature  component  masking  a  less  mature  one  in  the  metric.  MRL 
normalizations  and  definitions  are  shown  in  Table  2.  Note  that  the  normal¬ 
ized  MRL  (mrl)  does  not  replace  the  existing  MRL,  but  is  an  intermediate 
step  needed  for  SRL  calculation  as  is  the  normalized  IRL  (irl). 


TABLE  2.  NORMALIZED  MANUFACTURING  READINESS 

LEVEL  DEFINITIONS 

MRL  = 

j 

MRL  for  technology  j 

mrl  = 

j 

normalized  MRL  for  technology  j 

mrl  = 

MRL/m* 

Research 

level 

6.1  6.2  6.3  6.4  6.5 

6.6 

6.7 

m* 

2  3  4  5  6 

8 

10 

The  SRL  metric  is  formed  by  multiplying  the  normalized  MRL,  the  TRL, 
and  the  mean  of  the  normalized  IRL  in  a  scalar  contraction  (dot  product) 
such  that  each  component  SRL .  has  a  value  from  1  to  TRL  as  does  the  scalar 
SRL.  System  readiness  definitions  and  nomenclature  are  shown  in  Table  3. 
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TABLE  3.  SYSTEM  READINESS  LEVEL  DEFINITIONS 
AND  NOMENCLATURE 


_  System  readiness  level  scalar,  the  mean  of  the  system  readiness 
levels  for  all  component  technologies 

SRL  =  System  readiness  level  for  component  j 


SRL  =  mrliTRLjirlj 

SRL  =  mean[{mr/^  mr/2 ...  mr/n}.{TRL^  TRL2 ...  TRLn}.{ir/^  ir/2 ...  ir!n}~\ 


Numerical  Examples 

For  simplicity  and  clarity,  this  article  shows  three  numerical-only 
examples  using  a  hypothetical  system  with  three  technologies,  as  shown 
in  Figure  6. 


Each  technology  has  an  associated  TRL  and  MRL.  Each  Interface  has  an 
associated  IRL.  Notationally,  this  will  be  of  the  form  shown  in  equations 
(6),  (7),  and  (8). 


TRL.  =  {TRLX,  TRL2,  TRL3} 


(6) 


MRL.  =  {MRLX,  MRL2,  MRL3}. 


(7) 
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X 

IRN  'j 

IRL..  =  h 

U 

X 

X 

X 

Early  Technology  with  Adequate  Interfaces 

The  author  assumes  a  simple,  three-component  system,  with  com¬ 
ponents  1,  2,  and  3  funded  at  the  6.2— applied  research  level— using  the 
following  values  for  MRL,  TRL,  and  IRL  shown  in  equations  (9),  (10),  (11), 
and  (12). 


MRL.={2,3,3} 

TRL.={3,2,4} 


TRL  =  mean[ 3,2,4]  =  3 


X 

IRN 

=  2 

II 

Ph 

1 — 1 

X 

X 

=  2 

X 

X 

>■ 


(9) 

(10) 

(ID 


(12) 


The  first  step  is  to  calculate  the  normalized  mrl  and  irl  using  the  equations 
from  Tables  1  and  2.  Because  this  is  6.2  funded,  the  m *  normalization  factor 
is  3  from  Table  2,  indicating  that  we  expect  6.2  funded  technologies  to  be  at 
an  MRL  of  3  before  progressing.  Likewise,  the  i*  normalization  factor  is  2 
from  Table  1.  Normalized  values  are  shown  in  equations  (13),  (14),  and  (15). 

mrl=MRL./m^{2,3,3}/3={0.66,l,l} 

(13) 


irl..  =  IRL../2  =  <1 

ij  ir 


0.5 

1  'I 

X 

1 

X 

(14) 
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irlx  -  mean[irl12,  irlj  =  0.75 ,  irl2  =  mean[irl12,  irl23]  =  0.75,  irl3  = 
mean[irl23,  irl  J  =  1. 


(15) 


The  SRL  vector  is  calculated  from  the  products  of  the  normalized  mrl, 
average  normalized  irl,  and  TRL  vectors  using  the  formulas  from  Table  3, 
shown  in  equations  (16)  and  (17). 

SRL.  =  mrl.  TRIM.  =  {0.66*3*0.75,1*2*0.75,1*4*1}  =  {1.49,1.5,4} 

(16) 


SRL  =  mean[SRL.]  =  2.33 


(17) 


Analysis 

SRL  =  2.33  while  the  average  TRL  is  3,  indicating  a  slight  lag  in  at  least 
one  interface.  From  the  normalized  MRL,  one  can  conclude  that  the 
system  is  at  a  mostly  appropriate  level  of  manufacturing 
readiness  with  two  components  at  an  mrl  of  1  and 
one  at  0.66.  SRL1  and  SRL2  are  at  1.5,  slightly  lag¬ 
ging  behind  the  technology  readiness  of  2  and  3 
due  to  some  interface  development  that  needs  to 
occur.  SRL3  =  TRL3  =  4  indicates  that  this  tech¬ 
nology  is  at  an  appropriate  level  of  interface 
and  manufacturing  readiness.  The  metric  indi¬ 
cates  to  management  that  it  is  time  to  devote 
additional  resources  to  the  interfaces  of 
technologies  1  and  2  before  pushing  ahead  in 
further  component  or  system  development. 

It  is  very  important  to  conduct  the  early 
phases  of  interface  readiness,  which 
involve  subject  matter  experts  from  dif¬ 
ferent  fields  exchanging  information  and 
ensuring  that  there  exists  an  interface 
solution.  If  this  is  skipped,  then  at  the 
demonstration  and  prototyping  levels 
of  6.4  research,  many  technology  choices 
must  be  revisited  because  the  technologies  have 
matured  separately  and  are  becoming  incompatible.  Revisiting 
technology  choices  may  then  result  in  program  delays,  cost  overruns,  or 
mad  scrambles  to  prepare  for  demonstrations  or  program  cancellations. 
The  classic  case  of  this  is  thermal  management,  when  a  new  technology 
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becomes  available  with  thermal  management  as  an  afterthought,  and  the 
legacy  platform  for  which  it  is  intended  becomes  overwhelmed  with  the 
new  thermal  load.  The  thermal  issues  associated  with  5th  generation  air¬ 
craft  (Majumdar  &  Kjelgaard,  2015)  are  a  result  of  thermal  interface  as  an 
afterthought.  The  opposite  appears  to  be  happening  in  the  semiconductor 
industry  in  which  thermal  management  is  a  very  active  area  of  research  in 
anticipation  of  higher  thermal  loads  on  microchips  in  the  near  future. 

Mid-level  Technology  with  Lagging  Interfaces 

In  a  more  abbreviated  form  than  the  previous  example,  we  assume  a 
simple,  three-component  system  with  components  1,  2,  3  funded  at  the 
6.4  -  demonstration  level.  MRL.  =  {3,5,5};  TRL.  =  {6,4,5};  TRL  =  5,  and  the 
IRL  has  the  following  values:  IRL12  =  3,  IRL13  =  2,  and  IRL23  =  2.  The  SRL 
calculation  is  as  follows: 

mrl.  =  {0.6, 1,1}.  irl12  =  3/5,  irl13  =  2/5,  and  irl23  =  2/5,  irl1  -  mean[irll2,  irll3]  = 
0.5 ,  irl2  =  mean[zrZ12,  irl2^\  =  0.5,  irl3  =  mean[zrZ23,  zrZJ  =  0.4. 

SRL.  =  mrl.  TRL.  irl.  =  {0.6*6*0.5,1*4*0.5,1*5*0.4}  =  {1.8, 2, 2} 

SRL  =  mean[SRL.]  =  1.3 


It  makes  no  sense  to  continue  and  pursue  more  ma¬ 
ture  technology  that  may  or  may  not  work  in  the 
intended  environment  or  with  the  other  subsystems. 


Analysis 

The  fact  that  SRL  =  1.3,  but  there  are  TRLs  at  6  and  4  and  an  average 
TRL  of  5,  alerts  management  there  are  serious  manufacturing  and  interface 
issues,  probably  due  to  neglect  in  early  technical  development.  Note  that  the 
mr l±  is  0.6  and  is  slightly  lower  than  the  other  two;  the  SRL.  are  very  nearly 
all  at  2;  and  the  TRL.  are  quite  high— at  6, 4,  and  5 —due  to  the  irl  being  much 
lower.  This  alerts  management  that  emphasis  needs  to  be  placed  on  develop¬ 
ing  interfaces.  Further  component  maturation  is  very  risky  and  very  likely 
counter-productive.  It  makes  no  sense  to  continue  and  pursue  more  mature 
technology  that  may  or  may  not  work  in  the  intended  environment  or  with 
the  other  subsystems.  This  system  is  headed  toward  program-killing  safety, 
thermal,  control,  electrical,  or  other  integration  and  deployment  issues. 
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Advanced  Technology  with  One  Lagging  Interface 

The  author  assumes  a  simple,  three-component  system,  with  compo¬ 
nents  1, 2,  and  3  funded  at  the  6.7  —  operational  systems  development  level. 
MRL.  =  {7,7,7};  TRL.  =  {7,7,7}  and  the  IRL  has  the  following  values:  IRL12  =  7, 
IRL13  =  7,  and  IRL23  =  4.  The  SRL  calculations  are  as  follows: 

mrl  ={1,1,1}.  irl12  =  1,  irl13  =  1,  and  irl23  =  0.57,  irlx  -  mean [irl12,  zrZ13]  =  1 ,  irl2  = 
mean[zrZ12,  irZ23]  =  0.79, ,  irl3  =  mean[zrZ23,  irl 13]  =  0.79. 

SRL.  =  mrl.  TRL.  irl.  =  {1*7*1,1*7*0.79X7*0.79}  =  {7, 5.5, 5.5} 

SRL  =  mean[SRL.]  =  6 

Analysis 

SRL  =  6,  but  the  TRLs  are  all  at  7.  This  alerts  management  that  there  is 
at  least  one  interface  or  manufacturing  issue.  Examining  the  component 
SRLs  reveals  that  SRLX  =TRLX  =  7,  but  the  other  two  SRLs  lag  TRL,  indi¬ 
cating  that  the  interfaces  from  component  2  to  3  are  lagging  and  should  be 
addressed  before  developing  the  component  technologies  further. 

Practical  Example— High  Energy  Laser  System 

Note:  This  is  an  example  and  not  representative  of  any  particular  sys¬ 
tem.  A  high  energy  laser  system  is  in  early  research  and  development, 
primarily  funded  by  6.2  and  6.3  sources.  It  is  composed  of  at  least  the  fol¬ 
lowing  subsystems:  laser,  beam  director  (BD),  thermal  management  (TM), 
electrical  management  (EM),  structural  support  (Struct),  atmospheric 
propagation  (Atmos),  target,  target  acquisition,  tracking,  pointing  (ATP), 
and  battle  management  and  controls  (BM).  A  TRL  assessment  might  be  as 
follows  (Table  4). 


TABLE  4. 

,  SAMPLE  TRL/MRL  RATINGS 

Subsystem 

Laser 

O 

CD 

TM 

z 

111 

Struct 

Atmos 

Target 

ATP 

BM 

Controls 

TRL 

4 

4 

3 

3 

2 

4 

n/a 

3 

2 

3 

MRL 

4 

4 

4 

4 

2 

3 

n/a 

2 

2 

2 

Note  that  the  target  TRL  and  MRL  are  ‘ 

n/a*  because  the  system  does  not 

involve  building  the  target,  but  the  atmosphere  and  ATP  form  an  interface 
with  the  target  so  an  IRL  is  associated  with  the  target,  but  no  TRL. 
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An  IRL  matrix  might  look  as  shown  in  Table  5,  if  most  effort  had  been  placed 
into  developing  the  laser,  beam  director,  ATP  algorithms;  and  target  infor¬ 
mation,  but  not  much  effort  placed  on  'system'  issues,  such  as  the  electrical 
or  thermal  management  systems  or  the  controls  architectures.  For  simplic¬ 
ity,  only  the  upper  half  of  the  matrix  is  shown  assuming  that  IRL..  =  IRL... 


TABLE  5.  SAMPLE  IRL  RATINGS 


Not  every  component  has  every  kind  of  interface  so  that  the  n/a  values  in 
Table  5  are  simply  not  part  of  the  calculation.  Applying  the  equations  in 
Tables  1,  2,  and  3  yields  the  results  shown  in  Table  6,  assuming  normaliza¬ 
tion  by  the  6.3  funding  values  from  Table  1  and  Table  2.The  SRL  =  2.19.  The 
average  TRL  =  3.1. 


TABLE  6.  SAMPLE  IRL,  MRL  AND  SRL  COMPONENTS 


£ 

o> 

+■* 


>* 

V) 

SI 

3 

0) 

</> 

ro 

O 

Z 

Z 

u 

3 

•M 

o 

E 

4-» 

Q) 

U\ 

k. 

ic 

0. 

H 

Z 

4-* 

c 

o 

C/> 

-1 

CO 

H 

LU 

t/> 

< 

1- 

< 

CD 

u 

irli 

0.67 

0.81 

1.1 

1.14 

1.13 

0.71 

0.75 

0.91 

0.86 

1.05 

mrl. 

1 

1 

1 

1 

0.5 

0.75 

0.5 

0.5 

0.5 

SRL 

2.67 

3.26 

3.29 

3.43 

1.13 

2.13 

1.36 

0.86 

1.57 
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The  average  SRL  is  below  the  average  TRL,  indicating  that  there  are  some 
integration  or  manufacturing  issues  that  should  be  addressed  before  the 
components  are  developed  further.  Specifically,  the  laser  subcomponent 
itself  is  a  TRL  4,  with  an  appropriate  level  of  manufacturability;  however, 
its  average  IRL  is  the  lowest  of  any  of  the  other  subsystems.  Such  a  system 
is  in  danger  of  developing  a  main  component  that  cannot  be  integrated, 
demonstrated  with  a  prototype  system  at  an  appropriate  level,  or  that 
will  come  up  with  extensive  integration  issues  late  in  development.  These 
integration  issues  may  prove  to  be  very  costly  and  time-consuming  to  fix. 
It  would  be  best  to  develop  the  laser-thermal,  laser-electrical,  laser-battle 
management,  and  laser  control  interfaces  before  continuing  to  mature  the 
laser  technology  itself.  The  side  benefit  would  be  the  ability  to  demonstrate 
early  prototype  laser  systems  rather  than  waiting  for  full  maturity  of  the 
final  laser  to  conduct  any  demonstrations,  which  would  be  conducive  to 
maintaining  the  interest  in  funding  this  technology  development  effort. 


Verbal  System  Readiness  Level 
Definitions 

The  proposed  mathematical  definition  of  SRL  permits  a  verbal  defini¬ 
tion  of  SRLs  in  a  way  that  the  Sauser  definition  and  mathematics  did  not. 
There  is  one  caveat  to  these  verbal  definitions:  they  strictly  hold  fast  at  those 
milestones  of  development  where  SRL=TRL.  It  is  possible  to  have  an  SRL  of 
3  with  TRLs  of  6  by  ignoring  interfaces  and  manufacturing,  in  which  case 
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the  following  definition  of  SRL  =  3  (Table  7)  would  not  be  accurate  because 
the  SRL  metric  is  significantly  lagging  the  TRL  metrics.  This  caveat  also 
helps  ensure  that  one  cannot  inappropriately  claim  a  high  level  of  SRL  by 
having  one  mature  component  mask  a  less  mature  one. 


TABLE  7.  SYSTEM  READINESS  LEVEL  DEFINITIONS  ] 

SRL 

Name 

Definition 

1 

System  concept 

The  system  concept  has  been  identified 
to  include  the  subsystems.  Overall  system 
functional  requirements  are  qualitatively 
understood. 

2 

System 

technologies 

Subsystem  technology  path  identified  to  include 
a  specific  technology  solution.  Technology, 
manufacturing,  and  interface  drivers  understood. 

3 

System  proof  of 
concept 

Experimental  evidence  has  been  obtained  that 
the  system  is  possible  in  principle  to  develop  and 
manufacture. 

4 

System 

component 

verification 

All  system  components  have  been  built 
and  tested  in  a  laboratory  environment 
separately.  Numerical  studies  show  component 
compatibility. 

5 

System 

component 

validation 

All  system  components  have  been  built  and 
tested  in  a  relevant  or  emulated  production  and 
deployment  environment.  Components  with 
simulated  interfaces  have  been  tested. 

6 

System 

prototype 

demonstration 

A  system  prototype  has  been  demonstrated  and 
fabricated  in  a  relevant  environment.  Interface 
control  has  been  demonstrated  traceable  to  a 
deployed  environment. 

7 

System 

operational 

demonstration 

An  integrated  system  prototype  has  been 
demonstrated  and  fabricated  in  an  operational  / 
manufacturing  environment. 

8 

Actual  system 
demonstration 

The  production  representative  system  has  been 
demonstrated  in  an  operational  environment. 

9 

Operational 

system 

Production  system  is  used,  demonstrated,  and 
maintained  in  an  operational  environment. 

Generalized  Integration  Readiness  Level  Definitions 

The  author  proposes  the  simplified  critical  item  lists  (Table  8)  for  the 
IRLs  (U.S.  Navy,  2009).  The  simplified  lists  allow  a  wider  application  to 
physical  rather  than  IT  systems,  and  focus  on  the  few  truly  critical  mile¬ 
stones  rather  than  many  contributing  factors.  See  U.S.  Navy  (2009,  p.  6)  for 
a  comparison. 
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TABLE  8.  SIMPLIFIED  INTEGRATION  READINESS  LEVEL  DEFINITIONS 

IRL  Name 

(Sauser  et  al., 

Critical  Items 

2010)  Definition 

An  Interface 

• 

There  exists  a  functional  flow  block 

c 

between 

diagram  for  the  technology  and 

-4— * 

technologies  has 

its  interfaces  in  a  notional  system 

(T3 

i  ■- 

been  identified 

concept. 

1  M— 

■4— ' 

with  sufficient 

• 

Subject  matter  experts  for  each  of 

c 

CD 

detail  to  allow 

the  connecting  technologies  have 

~o 

characterization  of 

been  identified  and  a  technical 

the  relationship. 

interchange  held. 

There  is  some 

• 

Input  and  output  parameters  have 

c 

o 

level  of  specificity 

been  identified  for  each  interface. 

4— » 

CD 

to  characterize 

N 

the  Interaction 

2  cd 

(i.e.,  ability  to 

o 

CD 

influence)  between 

CD 

(— 

technologies 

U 

through  their 

interface. 

There  is 

• 

Parametric  or  physics-based 

>, 

Compatibility 

models  describe  the  interface  at  the 

A-> 

(i.e.,  common 

qualitative  level  so  that  the  impact 

!o 

language)  between 

on  each  of  the  identified  parameters 

3  1 

technologies 

can  be  modeled  at  the  system  level. 

E 

to  orderly  and 

• 

Interface  risks  have  been  identified. 

o 

efficiently 

• 

Interface  constraints  have  been 

integrate  and 

identified. 

interact. 

There  is  sufficient 

• 

A  solution  space  exists  to  meet 

CD 

L.  r  i 

detail  in  the 

design  concept  requirements. 

ro  £ 

Quality  and 

• 

Generic  interface  models  have  been 

4  j?  2 

Assurance  of 

validated  by  experiment. 

if 

the  integration 

O  < 

between 

technologies. 

There  is  sufficient 

• 

Interfaces  are  well  defined. 

Control  between 

• 

Interfaces  have  been  demonstrated 

O 

technologies 

in  a  laboratory  environment. 

5  c 

necessary  to 

• 

Specific  interface  models  have  been 

o 

u 

establish,  manage, 

validated  by  experiment. 

and  terminate  the 

integration. 
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TABLE  8, 

CONTINUED  ! 

IRL  Name 

(Sauser  et  al., 

2010)  Definition 

Critical  Items 

CD 

Information 

The  integrating 
technologies  can 
Accept ,  Translate, 
and  Structure 
Information  for 
their  intended 
application. 

Control  architecture  is  developed. 
Software  components  work 
together. 

Individual  modules  are  tested 
with  control  signals  to  verify 
performance. 

Integrated  system  demonstrations 
are  completed. 

c  The  integration 
0+3  of  technologies 
■Jo  tj  has  been  Verified 
£  Id  and  Validated  with 
^  >  sufficient  detail  to 
>  c  be  actionable. 

fD 


Fully  integrated  prototype  in 
simulated  operational  environment. 
Each  interface  tested  under  stressed 
and  anomalous  conditions. 


Actual  integration 
-g  completed  and 
o  Mission  Qualified 
8  $  =  through  test  and 

if  ^  demonstration, 
in  the  system 
environment. 


System  is  fully  integrated  in  an 
operational  environment. 

All  flight  and  safety  qualifications 
are  completed  for  all  technologies 
and  interfaces. 

Form,  fit,  and  function  are  verified. 


Integration  is 
§  §  Mission  Proven 
9  w  >  through  successful 
if  cf  mission  operations. 


System  is  fully  integrated  and 
has  demonstrated  operational 
effectiveness. 

Interface  failure  rates  are  fully 
characterized. 


Use  of  the  SRL  Metric 

Any  time  the  performance  or  behavior  of  a  complex  system  is  summa¬ 
rized  by  a  single  number,  there  is  inevitable  loss  of  information  and  the 
potential  for  false  indication.  SRL  and  IRL  have  a  subjective  component  to 
them,  as  do  TRL  and  MRL.  The  existence  of  the  SRL  metric  will  not  com¬ 
pletely  compensate  for  organizational  or  programmatic  pressure  to  advance 
technologies  prematurely  to  meet  budget  and  schedule.  It  will,  however,  fos¬ 
ter  an  awareness  of  the  cost  of  doing  so.  The  SRL  metric,  as  defined  herein, 
is  designed  to  be  an  indication  that  a  system  or  component  is  ready  for  the 
next  step  in  development  when  the  system  readiness  is  commensurate  with 
the  technology  readiness.  From  equations  (5)  and  (11),  where  SRL  =  TRL  at 
the  system  level  and  SRL.  =  TRL.  at  the  component  level,  advancement  is 
appropriate.  Since  interfaces  cannot  be  more  mature  than  their  component 
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technologies,  SRL  will  lag  TRL  at  each  step  of  development.  At  that  point, 
the  program  will  move  to  a  higher  funding,  maturity,  or  development  cat¬ 
egory;  the  normalization  factors  will  change;  and  SRL  will  once  again  lag 
TRL  as  shown  conceptually  in  Figure  7.  This  built-in  safeguard  will  reduce 
the  possibility  of  a  mature  subset  of  the  system  overshadowing  a  less  mature 
part  and  giving  false  indications  (Kujawski,  2010).  A  further  safeguard  can 
be  implemented  by  limiting  the  values  of  the  normalized  IRLs  and  MRLs 
( mrl  and  irl  from  Tables  1  and  2)  to  a  maximum  of  1.0,  further  ensuring  that 
one  mature  component  cannot  mask  a  less  mature  one.  The  principle  that 
advancement  to  the  next  level  of  funding  or  acquisition  should  not  occur 
until  the  system  readiness  is  commensurate  with  the  technology  readiness 
can  and  should  be  applied  at  the  system  level  (when  SRL  =  mean[TRL.])  and 
at  the  component  technology  level  (when  SRL.  =  TRL.). 


Conclusions 


This  article  has  proposed  a  modification  to  the  Sauser  mathematics  of 
IRL  and  SRL  that  allows  an  SRL  metric  that  gives  a  clear  indicator  of  when 
a  component  technology  or  system  is  ready  for  further  advancement  and 
allows  for  standard  verbal  definitions  of  SRL.  SRL  and  IRL  need  to  be  incor¬ 
porated  into  the  system  engineering  process  early  in  development.  TRL  has 
been  a  valuable  metric;  however,  its  lack  of  emphasis  on  systems  issues  has 
resulted  in  divergent  development,  where  some  system  components  are 
developed  beyond  their  interfaces  and  manufacturing,  resulting  in  legacy 
decisions  that  impede  demonstration  and  integration.  A  useful  SRL  metric 
can  help  to  foster  more  balanced  and  cost-effective  technology  development. 
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Appendix 

Standard  Technology  Readiness  Level  and  Manufacturing 
Readiness  Level  Definitions 


TABLE  A-1.  STANDARD  TRL  DEFINITIONS  1] 

TRL 

Definition 

1 

Basic  principles  observed  and  reported 

2 

Technology  concept  and/or  application  formulated 

3 

Analytical  and  experimental  critical  function  and/or  characteristic 
proof  of  concept 

4 

Component  and/or  breadboard  validation  in  a  laboratory 
environment 

5 

Component  and/or  breadboard  validation  in  a  relevant  environment 

6 

System/subsystem  model  or  prototype  demonstration  in  a  relevant 
environment 

7 

System  prototype  demonstration  in  an  operational  environment 

8 

Actual  system  completed  and  qualified  through  test  and 
demonstration 

9 

Actual  system  proven  through  successful  mission  operations 

Note.  (DoD,  2011) 
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TABLE  A-2.  STANDARD  MRL  DEFINITIONS 

MRL 

Definition 

1 

Basic  manufacturing  implications  identified 

2 

Manufacturing  concepts  identified 

3 

Manufacturing  proof  of  concept  developed 

4 

Capability  to  produce  the  technology  in  a  laboratory  environment 

5 

Capability  to  produce  prototype  components  in  a  production¬ 
relevant  environment 

6 

Capability  to  produce  a  prototype  system  or  subsystem  in  a 
production-relevant  environment 

7 

Capability  to  produce  systems,  subsystems,  or  components  in  a 
production-representative  environment 

8 

Pilot  line  capability  demonstrated;  ready  to  begin  Low  Rate  Initial 
Production 

9 

Low  Rate  Initial  Production  demonstrated;  capability  in  place  to 
begin  Full  Rate  Production 

10 

Full  Rate  Production  demonstrated  and  Lean  production  practices 
in  place 

Note.  (DoD,  2012) 

TABLE  A-3.  DoD  STANDARD  FUNDING  CATEGORIES 

6.1 

Basic  Research 

6.2 

Applied  Research 

6.3 

Advanced  Technology  Development 

6.4 

Advanced  Component  Development  and  Prototypes 

6.5 

System  Development  and  Demonstration 

6.6 

RDT&E  Management  Support 

6.7 

Operational  Systems  Development 

Note.  (DAU,  2016) 
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